Advances in Neural Networks

PREFACE Artificial neural networks or simply neural networks represent an emerging technology rooted in many disciplines. This popular and important area of science and technology was extensively developing for the recent period of time. Neural networks are endowed with some unique attributes, like the ability to learn from and adapt to their environment and the ability to approximate very complicated mappings. We will consider here some basic principles of neural networks and several novel ideas related to them. After a brief historical observation we will consider, what a neuron is, and what a neural network is. We will observe the basic ideas of the threshold logic and thus, the threshold neuron (the classic perceptron) will be considered. It will be observed that historically the first neural networks were developed for the implementations of the non-threshold Boolean functions. We will consider the basic principles of the learning process and the error-correction learning, in particular. A classical network-a feedforward neural network and its backpropagation learning will be presented. Then we will move to several novel and advanced solutions in neural networks that are the main subject of this course. We will consider two types of the complex-valued neurons, whose activation functions are functions of the argument of the weighted sum. There are the universal binary neuron, which can implement non-threshold Boolean functions and the multi-valued neuron, which can implement those mappings that are described by the threshold functions of multiple-valued logic and by the continuous multiple-valued functions. The learning process for both neurons will be considered. A special attention will be paid to the implementation of the nonlinearly separable Boolean functions including the XOR and parity n ones on the single universal binary neuron. We will also consider a feedforward neural network based on multi-valued neurons and its backpropagation learning, which does not require a derivative of the activation function. Cellular neural networks, their architecture, their different types and their applications in image processing will be studied in the concluding part of the course. The author is deeply indebted to Professor Claudio Moraga, who has given freely of his time to read through the Class Notes, for his very useful and helpful comments.